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(57) Abstract 

A machine independent high resolution digital screen recorder (12) is disclosed for providing high quality video displays with 
manageable storage capacity and bandwidth. The screen recorder includes an analog to digital frame grabber (24) for converting a high 
resolution video signal (20) that modulates a video display (14) into RGB sequences of digital frames (30). A video compression unit (26) 
separates the high and low variance portions of the digital frames, encodes them with respective lossy (56) and lossless (54) compression 
algorithms and stores them in a mass storage device (28). 
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HIGH RESOLUTION DIGITAL SCREEN RECORDER AND METHOD 

BACKGROUND OF THE INVENTION 
Field of the Invention 

The present invention generally relates to the field 
of screen recorders, and more specifically to a high reso- 
5 lution digital screen recorder and recording method that 
converts an analog RGB video signal into a compressed digi- 
tal signal. 

Description of the Related Art 

10 Conventional color televisions and display monitors 

produce relatively low resolution and bandwidth red, green 
and blue (RGB) video signals, e.g. 400-600 lines per, frame 
at 25-30 frames per second, that conform to one of several 
international standards. Three of the most common stan- 

15 dards are: the National Television Systems Committee (NTSC) 
standard with 525 lines per frame at 30 frames per second, 
the Sequential Chrominance Signal & Memory (SECAM) standard 
with 625 lines per frame at 25 frames per second and the 
Phase Alternating Line (PAL) standard with 625 lines per 

20 frame at 25 frames per second. These standards are fairly 
flexible in that the number of scan lines actually used by 
a display may be significantly less than the specified sta- 
ndard. For these low resolution signals, video recording 
devices, such as video cassette recorders (VCRs) , record 

25 the analog signal directly onto a magnetic tape. 

For high resolution workstation monitors having 1280 
or greater scan lines per frame, or high definition televi- 
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sion (HDTV) , , it is not currently feasible to record the 
analog video signals directly onto the tape in an analog 
format at the same high resolution • The storage require- 
ments and bandwidth of the video signals exceed the capa- 
5 bilities of analog recording devices. For example, a 1280 
x 1024 video signal at 3 bytes per pixel and 30 frames per 
second recorded for 3 hours would require 1274 Gbytes of 
memory and a bandwidth of 943 Mbps. As a result, in a cur- 
rent approach a scan converter is used to convert the high 

10 resolution video signal into one of the low resolution sta- 
ndard formats, which is recorded onto the tape. The down 
conversion is done by averaging adjacent scan lines or sim- 
ply skipping scan lines, and results in a substantial loss 
of resolution when the recorded tapes are played back. The 

15 "RGB/Videolink 1600U", RGB Spectrum of Berkeley, Califor- 
nia, 1991 is described in a product bulletin as one example 
of a scan converter. The loss of resolution is particular- 
ly bothersome in a multimedia workstation, a typical dis- 
play includes text, graphics, pictures, audio and motion 

20 picture video data. The graphic icons and text are dist- 
orted by converting the high resolution signal to the rela- 
tively coarse low resolution signal. This system sacrific- 
es image quality to maintain manageable storage require- 
ments and bandwidth. For applications where the replayed 

25 information is required to faithfully reconstruct events, 
e.g., air traffic control displays or is critical to liti- 
gation, the loss of dielity may make the recording unuse- 
able • 

30 SUMMARY OF THE INVENTION 

The present invention seeks to provide a machine inde- 
pendent, high resolution digital screen recorder that pro- 
vides high quality video displays with manageable storage 
capacity and bandwidth, and can be implemented as a stand- 

35 alone or integrated unit. 
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This is accomplished with a screen recorder for re- 
cording a high resolution analog multi-color video signal, 
preferably RGB, that modulates a video display. An analog 
to digital frame grabber converts the high resolution video 
5 signal into sequences of digital frames. A video com- 
pression unit encodes the sequences of digital frames into 
compressed digital signals and stores them in a mass stor- 
age device. 

In a preferred embodiment, a high resolution multime- 
10 dia computer workstation displays high variance (motion 
picture video) and low variance (graphics, text, icons and 
background) data and produces the video signal to modulate 
its display. The video compression unit separates the high 
and low variance portions of the digital frames and encodes 
15 them with lossy and lossless compression algorithms, re- 
spectively. 

For a better understanding of the invention, and to 
show how the same may be carried into effect, reference 
will now be made, by way of example, to the accompanying 
20 drawings. 

RRTEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram of a high resolution comput- 
er workstation and a digital screen recorder; 
25 FIG. 2 shows a typical multimedia display; 

FIG. 3 shows a difference frame for the display of 
FIG. 2; 

FIG. 4 is a flowchart of a hybrid video compression 
algorithm; 

30 FIG. 5 is a flowchart of the lossless compression al- 

gorithm ; 

FIG. 6 is a flowchart of the lossy compression algo- 
rithm; 

FIG. 7 is a flowchart of the video window detection 
35 algorithm; 
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FIGs. 8a-8d illustrate the steps of the detection al- 
gorithm of FIG. 7; and 

FIGs. 9a-9d show typical video windows, their projec- 
tions and corresponding transition codes. 

5 

PRTATT.EP DESC RIPTION OF THE INVENTION 

High resolution multimedia computer workstations oper- 
ate in a windowing environment that consists of several 
text and graphics windows, a motion video window, icons, a 
10 cursor and possibly audio signals, and in some applications 
live sensor images such as radar may also be present. A 
typical display has a resolution of at least 1280 x 1024 
pixels and is refreshed at a rate of 30 frames or higher 
per second, although the invention is applicable for arbi- 
15 trary resolutions and frame rates. FIG. 1 shows a high 
resolution multimedia computer workstation 10 that is elec- 
trically connected through a screen recorder 12 to a moni- 
tor 14. Typical workstations have a single video output 
for the monitor, and thus to display the video signal in 
20 real time and store it as a compressed digital signal for 
future playback, the monitor and screen recorder are con- 
nected in a "loopthrough" configuration that is similar to 
a TV/VCR connection. The workstation internally generates 
high resolution digital RGB video frames 16 in a machine 
25 dependent format in response to the windowing environment, 
graphics, text or motion video applications and uses a D/A 
converter 18 to convert them into an analog RGB video sig- 
nal 20. The frames 1 resolution and rate are included as 
sideband information in the video signal. The analog video 
30 signal 20 modulates the intensity of a cathode ray tube 
(CRT) 22 in the monitor 14 to raster scan the RGB images 16 
onto the display at the given frame rate. The screen re- 
corder 12 captures the video signal 20 and stores it as a 
compressed digital signal. 
35 The screen recorder can be a stand-alone, workstation 
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independent: unit as shown, or it can be integrated into the 
workstation. In the preferred embodiment, the screen re- 
corder is connected to only video and/or audio outputs of 
the workstations digital display system and does not re- 
5 quire any information from internal components of the dis- 
play. Thus, the invention is not dependent on the imple- 
mentation of any specific display and may be applied to a 
variety of devices. The screen recorder includes a high 
resolution frame grabber 24, a digital video compression 
10 unit 26 and a mass digital storage device 28. The frame 
grabber converts the analog RGB video signal 20 in real 
time into another sequence of RGB digital frames 30 in a 
machine independent format. A suitable frame grabber, 
HI*DEF 111, is described in a new product bulletin from 
15 IMAGRAPH of Chelmsford, Massachusetts and claims a 160 Mhz 
bandwidth for capturing standard and non-standard video 
signals with up to 16K x 16K resolution. The compression 
unit 26 compresses the video frames 30 at a ratio of app- 
roximately 250:1 with only slight visual degradation and 
20 maintains reasonable bandwidths and memory requirements. 
The compressed digital video signal is stored in the stor- 
age device 28, which can be a conventional hard drive, an 
optical drive or a portable device such as a digital tape. 
In the invention, the multimedia display is separated 
25 into high and low variance portions for the purpose of com- 
pressing the digital video signal 30. The high variance 
portion 32 is defined by a video window. 34 in which some 
type of motion picture video signal is being played, while 
the low variance portion 35 is everything else: graphics 
30 36, text 38, icons 40 and the background 42. FIG. 2 shows 
a typical multimedia display and FIG. 3 shows the differ- 
ence between successive displays. As shown in these fig- 
ures, the high variance data 32 and low variance data 35 
are differentiated by a stark disparity in their temporal 
35 (interframe) and spatial (intraframe) variances. Because 
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the motion picture video data»s spatial and temporal vari- 
ances are relatively high, at a given encoding rate the 
error in its reconstructed image will be greater than the 
reconstruction errors in the graphics, text, icons and 
5 background data. However, motion picture (high variance) 
imagery can be compressed such that encoding errors only 
slightly distort the image's visual quality or are com- 
pletely imperceptible. Conversely, the low variance data 
is highly correlated temporally and spatially and is thus 
10 easy to compress, but any encoding errors are easily visi- 
ble and distort the fine structure of the icons, graphics 
and text. Improved overall compression and reconstructed 
image quality is achieved by using separate lossy and loss- 
less encoding algorithms for the high and low variance da- 
15 ta, respectively. In a perfect lossless algorithm the de- 
compressed images equal the digital images 30 provided by 
the frame grabber without distortion or error, while in a 
lossy algorithm the decompressed algorithm has some amount 
of distortion or error relative to the original. 
20 FIG. 4 is a flowchart of the hybrid video compression 

algorithm* To achieve the necessary speed, the video com- 
pression unit 26 implements the algorithm in hardware, al- 
though future advances in computer speed may allow a soft- 
ware implementation. In step 44, the frame grabber 24 pro- 
25 vides the compression unit with 24-bit RGB images 30 at 8 
bits per pixel for each frame extracted from the video sig- 
nal 20. In the next step 46, the capture errors of the 
frame grabber are eliminated by masking off a number of the 
least-significant-bits (Isbs) , e.g., one Isb per image pix- 
30 el. Each successive set of RGB frames 30 is then transf- 
ormed in step 48, using a YUV transform to produce Y lumi- 
nance and UV chrominance components 49. The YUV transform 
is disclosed on pages 14 and 17 of a new product informa- 
tion sheet "Programmable Color Space Converter and Color 
35 Corrector", Brooktree Corporation of San Diego, California, 



PCTYUS95/05961 



7 

1990. An alternative or slightly modified transform denot- 
ed generally as an XYZ transform could be used to provide 
an X intensity (luminance) component. In the next step 50, 
the Y, U and V digital frames are subtracted from their 
respective successive YUV frames to form YUV difference 
images 51, which are integer valued and can be represented 
exactly by a digital codeword. In step 52, the video win- 
dow's boundary coordinates are extracted from the Y differ- 
ence image (see FIGs. 7-9 for details) and passed to the 
lossless and lossy compression algorithms. The window's 
boundary coordinates are also included in the sideband in- 
formation 53 sent to the decoder. The video window can be 
computed for each difference image or for some period in 
accordance with the requirements of the specific compres- 
sion algorithms. 

In step 54 the low variance portion of each successive 
digital frame is compressed using a lossless algorithm (see 
FIG. 5 for details) , and in step 56 the high variance por- 
tion is encoded using a lossy algorithm (see FIG. 6 for 
details) ; these two steps produce respective bitstreams. 
The accompanying audio track is compressed in step 58 and 
its bitstream is multiplexed with the compressed high and 
low variance video data and sideband information bitstreams 
in step 60. In step 62, the multiplexed digital data 63 is 
written to the storage device 28. 

To playback the digitally stored video signal, a de- 
coder 64 demultiplexes the data 63, performs the respective 
inverse transforms on the compressed high and low variance 
signals and audio signals and adds the decoded images back 
together to produce a sequence of reconstructed digital 
images 65. A D/A converter 66 converts the reconstructed 
images 65 into an analog RGB video signal 67 to playback 
the stored multimedia session. Depending on the display's 
content and the size of the video window, the compression 
algorithm can realize compressions of approximately 250:1 
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with no distortion in the low variance portions and only 
slightly perceptible distortion in the high variance motion 
picture portion. 

FIG. 5 illustrates a preferred embodiment of the loss- 
less compression algorithm for encoding the multimedia dis- 
play^ low variance portion 35. The lossless algorithm is 
initiated by encoding the first YUV digital frames 49 and 
thereafter encoding the YUV difference frames 51 computed 
in step 50. To decode the sequence of compressed frames, 
the first frame must be transmitted so that the successive 
frames can be recursively reconstructed by decoding the 
next difference frame and adding it to the previous frame. 
It may be desirable to reset the encoding algorithm period- 
ically by directly coding a YUV frame every n frames, e.g., 
200 frames, to prevent error propagation due to noise. 

Instead of completely removing the pixels in the video 
window and only encoding the low variance data pixels in 
the lossless algorithm, those pixels inside the video win- 
dow can be set equal to zero and encoded with the low vari- 
ance data. Since the pixels inside the window are all set 
to a constant value, their information content and effect 
on the compression of the frame is negligible. Alterna- 
tively, the windowed pixels could be removed and the algo- 
rithms modified to keep track of the video window. This 
approach could provide slightly better compression but 
would be substantially more complicated. 

In step 68 of FIG. 5, the algorithm checks the reset 
condition and selects either the YUV frames 49 or the dif- 
ference frames 51. In the former case, the pixel values 
for the respective YUV frames 49 inside the video window 34 
are set to zero (step 69) and each frame is compressed with 
the Joint Bi-level Image Group (JBIG) algorithm (step 70) 
to produce the encoded bitstream. The JBIG algorithm is 
described in the September 16, 1991 "CCIT Draft Recommenda- 
tion T.82 ISO/IEC Committee Draft 11544 Coded Representa- 
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tion of Picture and Audio Information-Progressive Bi-level 
Image Compression" . When the difference frames 51 are se- 
lected, the respective YUV difference frames* pixels inside 
the window are set to zero (step 71) and the frames are 
5 encoded using run length codes (step 72) to produce another 
bit stream. Run length codes are described in Jayant and 
Noll, Prentice-Hall, "Digital Coding of Waveforms", 1984, 
pp. 465-485. Other lossless algorithms could be employed 
in place of the JBIG and run length codes without departing 
10 from the scope of the invention, 

FIG. 6 is a flowchart of the lossy compression algo- 
rithm for the high variance portion 32 of the multimedia 
display. The Motion Pictures Expert Group (MPEG) algorithm 
described in "Information Technology - Generic Coding of 
15 Moving Pictures and Associated Audio, Recommendation H.262, 
ISO/IEC 13818-2", November 25, 1993 is preferably used to 
compress the data inside the video window and the audio 
track, although other video compression algorithms are also 
applicable. MPEG reads 16 x 16 blocks of pixel data and 
20 processes groups successive frames, e.g., 12-15. To accom- 
modate the standard, the coordinates of the video window 
are updated for the first image of each group (FIG. 2, step 
52) and modified in step 74 to extend the window to fit the 
MPEG block size. In step 75, the YUV images 49 are conv- 
25 erted into a 4:2:0 format to be compatible with the MPEG 
algorithm. For every 2x2 block of pixels the Y luminance 
value is the value of each pixel and the U and V values 
equal the average of the four pixels from their respective 
frames. In step 76, the 4:2:0 formatted pixels inside the 
30 extended window are encoded with the MPEG algorithm to pro- 
duce the bitstream. The decoder strips out the additional 
pixels prior to reconstructing the display. 

FIG. 7 is a flowchart of a suitable video window de- 
tection algorithm, which utilizes the standard rectangular 
35 geometry of the workstation's windowing environment and the 
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contrast in signal characteristics between the motion pic- 
ture and graphics, text, icons and background portions of 
the display. In step 78, the algorithm receives the Y 
difference image. An activity measure, e.g., the variance, 
5 is computed in step 80 for each row and each column, with 
the variances 81, 82 projected onto the vertical and hori- 
zontal axes, respectively. The variance of each row or 

- « 

column is given by: o 2 -— £ (l^m) 2 where o 2 is the variance, 

l t is the pixel luminance value, m is the mean and n is the 

10 number of pixels in the row or column. The mean m of the 
difference images is normally approximately zero. The 
variance projections have sharp transitions at the video 
window f s edge coordinates 83, 84, 85, 86 and at the interi- 
or coordinates 87, 88 where the window 1 s width with respect 

15 to the particular axis changes, as shown in FIG. 8a. The 
coordinates of a bounding rectangle 89 for the video window 
34 are extracted from the first (83,85) and last (84,86) 
transitions of each projection (step 90) and specify the 
size and location of the window, as shown in FIG. 8b and 

20 denoted by a "0 M . In step 91, the interior row transition 
coordinates (87) are matched to the interior column transi- 
tion coordinates (88) to specify the interior corner points 
92, as shown in FIG. 8c and denoted by an W X W . 

To identify the window's shape, the projections 1 tran- 

25 sitions are coded (step 93) with a 10-digit transition code 
94. The code's format restricts the number of transitions 
to five or less in each projection 81 and 82; any projec- 
tion with more than five will default to the bounding rect- 
angle. A transition from low to high is indicated by a "1" 

30 and from high to low as a "0". Since the first transition 
of each projection is always a M l" and the last is always 
a w 0", only the interior transitions are coded. The first 
two flags specify the number of row projection interior 
transitions (0-3), flags 3-5 specify the transitions, flags 
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6-7 specify the number of column projection interior tran- 
sitions and flags 8-10 specify the transitions • Any unused 
flags in positions 3-5 and 8-10 are designated by an "X". 
For example, a simple rectangular window would be coded 
5 "OOXXXOOXXX". 

In step 95, a look-up-table (LOT) which contains the 
codes for a number of common shapes outputs a shape identi- 
fier 96 and a set of instructions 97 that specify how the 
interior corner points 92 are used to define the video win- 
10 dow's boundary coordinates 98, as shown in FIG. 8d. If the 
code 94 isn't included in the LUT, the window defaults to 
the bounding rectangle 89. The boundary coordinates, inte- 
rior points and shape identifier are sent to a buffer 99 
that holds the video window descriptors for a few frames, 
15 e.g., 3. If the information video window is approximately 
constant over the frames, it is output to the compression 
algorithm. If a window is identified for only 1 or 2 
frames it is discarded. This prevents the opening of a 
graphics or text window or an icon, which would cause a 
20 large variance in one difference image, from causing the 
algorithm to mis identify a video window. 

FIGs. 9a-d illustrate four common video window shapes, 
their variance projections, boundary coordinates, transi- 
tion codes and interior points. FIG. 9a shows an upper- 
25 left occluded window 100 with row and column variance pro- 
jections 102 and 104 respectively, interior point 106, tra- 
nsition code 108 and boundary coordinates 110. FIG. 9b 
shows a video window 112 having a rectangular void in the 
middle with row and column variance projections 114 and 116 
30 respectively, interior points 118, transition code 120 and 
boundary coordinates 122. FIG. 9c shows a video window 124 
having a notch in its upper side with row and column vari- 
ance projections 126 and 128 respectively, interior points 
130, transition code 132 and boundary coordinates 134. 
35 FIG. 9d shows an irregularly shaped video window 136 with 
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row and column variance projections 138 and 140 re- 
spectively , interior points 142, transition code 144 and 
boundary coordinates 146. The bounding rectangle's coordi- 
nates, the interior points' coordinates and the window's 
shape identification are included in the sideband informa- 
tion. The decoder 64 is provided with a table that matches 
the window ID with the proper set of instructions for pro- 
cessing the interior points 92. This algorithm identifies 
the video window's size, location and shape and produces 
its boundary coordinates directly from the sequential dif- 
ference images without receiving any input from the workst- 
ation f s window manager. This approach makes the screen 
recorder a workstation independent device. 

In the described embodiment, the screen recorder is 
used in conjunction with a high resolution multimedia work- 
station to digitally record and compress work sessions 
while maintaining the original high resolution display. The 
screen recorder can be used for applications such as educa- 
tion/training, communication, archiving, investigation and 
product marketing. The screen recorder can also be used to 
digitally record the analog RGB video signal for high defi- 
nition television (HDTV) by making the video window encom- 
pass the entire display. For improved performance, a more 
advanced algorithm may be developed for separating the rel- 
atively constant and varying portions of the video signal 
throughout the display could be employed. 

While several illustrative embodiments of the inven- 
tion have been shown and described, numerous variations and 
alternate embodiment will occur to those skilled in the 
art. Such variations and alternate embodiments are contem- 
plated, and can be made without departing from the spirit 
and scope of the invention as defined in the appended 
claims. 
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CLAIMS 

1. A screen recorder (12) for recording an analog 
video signal (20) -that modulates a video display (14), said 
video signal having a resolution greater than a standard 
National Television Systems Committee (NTSC) resolution of 
525 lines per frame, characterized by: 

an analog to digital frame grabber (24) for con- 
verting the high resolution video signal into a sequence of 
digital frames (30) ; 

a video compression unit (26) for compressing the 
sequence of digital frames into compressed digital signals 
(63) ; and 

a storage device (28) for storing the compressed 
digital signals. 

2. The screen recorder of claim 1, wherein said 
screen recorder (12) is a portable , workstation independent 
unit that is electrically coupled to a workstation (10) to 

5 receive said video signal (20) . 

3. The screen recorder of claim 1, wherein said sto- 
rage unit (28) is a hard disk, an optical drive or a digi- 
tal tape. 

4. The screen recorder of claim 1, further compris- 
ing a high resolution multimedia computer workstation (10) 
that generates said high resolution video signal (20) , said 
signal including high and low variance data. 

5. The screen recorder of claim 4, wherein said com- 
pression unit (26) separates each digital frame into the 
high (32) and low (35) variance data and compresses each 
independently into compressed first and second digital sig- 

5 nals 63. 



WO 95/33336 



PCT7US95/05961 



14 

6. The screen recorder of claim 5, wherein said com- 
pression unit (26) generates said compressed first signal 
as an approximation of the high variance data and said com- 
pressed second signal as an exact representation of the 

5 digital low variance data. 

7. A method for recording an analog video signal 
having a resolution greater than a standard National Tele- 
vision Systems Committee (NTSC) resolution of 525 lines per 
frame, the method comprising: 

5 converting the analog video signal into a se- 

quence of digital frames (44) ; 

compressing the sequence of digital frames into 
compressed digital signals (48, 50, 52, 54, 56); and 

storing the compressed digital signals (60, 62). 

8* The method of claim 7, wherein a high resolution 
multimedia computer workstation (10) that displays high and 
low variance video data produces the video signal (20) , and 
said digital frame compression step, comprises: 
5 separating each digital frame into said high and 

low variance video data (52) ; 

compressing said high variance data with a lossy 
compression algorithm to form a first compressed digital 
signal (56) ; 

10 compressing said low variance data with a loss- 

less compression algorithm to form a second compressed dig- 
ital signal (54) ; and 

storing the compressed first and second digital 
signals (60, 62) . 
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